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Abstract 

In this paper, we propose an extension to mixed multidimensional constraints of the prob¬ 
lem of state and input constrained control introdnced in [7], where the admissible set, namely 
the subset of the state space where the state and input constraints can be satished for all 
times, was studied, with focus on its boundary. The latter may be divided in two parts, one 
of them being called barrier, a semipermeable surface. We extend this notion of barrier to 
the mixed case and prove that it can be constructed via a minimum-like principle involving 
the Karush-Kiihn-Tucker multipliers associated to the constraints and a generalised gradient 
condition at its endpoints. 

Keywords mixed state and input constraints, barrier, admissible set, nonlinear systems. 

1 Introduction 

This paper is an extension to mixed constraints of the paper [7], the latter paper being devoted 
to the study of the admissible set for a nonlinear system with pure state and input constraints, 
namely constraints described by functions that depend on the state only and on the input only. 
The admissible set consists of all the initial conditions for which there exists a control such that the 
constraints are satished for all times. Its boundary can be divided into two complementary parts, 
one of which is called the barrier, proven to satisfy a minimum-like principle, therefore allowing 
its construction. The barrier enjoys the special property called semi-permeability: if the state, 
initiating from the interior of the admissible set, crosses the barrier, then it is guaranteed that it 
will violate the constraints in the future. Moroever, if the state starts outside the admissible set, 
no admissible trajectory can cross the barrier in the direction of the interior of the admissible set. 

In the current paper the above results are extended to the case where the constraints are mixed 
(see dH)) that is, they explicitly depend upon both the control and the state, without separation 
of these variables. Constraints of this type have been considered in the context of optimal control: 
for general theoretical results, the reader may refer to [IQIIIIIS] and, for applications where these 
constraints occur, to m in the context of tethered UAVs, or m in other aerospace applications. 

Note that, as opposed to the latter references, no a priori optimality notion in any sense is 
considered in this paper. More precisely, our approach may be applied to optimisation problems as 
a first step to prepare and simplify them by restricting the state space to the admissible set where 
an optimal solution, if any, may be found. 

Another important motivation to study mixed constraints is provided by flat systems [H 
[n], submitted to constraints: if we express the state and control variables in terms of a fiat 
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output y, namely x = ip{y,y,... and u = y, • ■ •, 2 /*-“"'’^^)) where y^^^ denotes the fcth 

order time derivative of y for an arbitrary integer fc, the constraint '){u) < 0 is transformed into 
7 (' 0 (y, y, ■. ■, 2 /^“"'’^^)) = 7 ( 2 /, y, ■ ■ ■, y^°‘\ i") £ 0 where v = yO+^) is the new control variable, the 
latter constraint described by 7 with respect to the transformed variables being naturally of a 
mixed nature. 

It turns out that the concepts of barrier and semi-permeability carry over to the mixed con¬ 
straint setting and that we can construct the barrier via a minimum principle, though containing 
significant modifications compared to the one of [7]. 

The main contribution associated with this generalisation is threefold: 

• Since the control u is only assumed measurable with respect to time, the evolution of the 
constraints along the integral curves of the system may be discontinuous and requires using 
tools from nonsmooth analysis [21 [S]. 

• Contrary to intuition, the equation satisfied by endpoints of the barrier is not gi{x,u) = 0 
for some i, x and u according to We prove that in fact the endpoints satisfy g{x) = 
min„g (7 maxi u) = 0 with additional generalised gradient conditions (Proposition |T3]). 

• To prove the minimum principle associated with the barrier (Theorem 15.11) . we use the 
same duality-like argument as in [7]: the boundary of the constrained reachable set at some 
time t, issued from any point of the barrier, is tangent to the barrier, and the respective 
normals of both boundaries are opposed. However, the characterisation of the extremum 
trajectories whose endpoints lie in the boundary of the reachable set, which constitutes 
the main step to prove the maximum principle, in the spirit of | 12 j . had to be generalised 
to the mixed constraint case (see Appendix El), assuming that the extremum control is 
piecewise continuous. This generalisation mainly consists in the construction of suitable 
needle perturbations that satisfy the constraints (Section EH) to generate the celebrated 
perturbation cone introduced by Pontryagin and coauthors |16] . a cone separated from the 
non-reachable part of the state space by a hyperplane whose normal is, at almost every 
instant of time, precisely the adjoint vector. 

The paper is organised as follows: in Section [2] the problem of characterising the admissible 
set in the mixed constraint case is presented along with the assumptions. Then we prove that this 
set is closed in Section [3] and study its boundary in Section |4] with an emphasis on the geometric 
properties of the barrier in Subsections 14.21 and 14.31 Then Section [S] is devoted to the derivation 
of the minimum principle associated with the barrier and is followed by examples in Section [51 
Final remarks and conclusions are presented in Section [7] and two appendices on the compactness 
of solutions and the maximum principle in the mixed constraint case are given in Appendix E and 
Appendix E respectively. 


2 Dynamical Control Systems with Mixed Constraints 

We consider the following constrained nonlinear system: 


( 1 ) 

( 2 ) 

(3) 

(4) 

where x{t) S 


X = f{x,u), 
x{to) = Xo, 
u C lA ■, 

gi{x{t),u{t)) < 0 for a.e. t G [to, 00 ) i = l,...,p 
being endowed with the usual topology of the Euclidean normf^ 


^We keep the same notation || • || for the Euclidean norm of for every p > 1. 
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We denote hy U a given compact convex subset of R"*, expressible as 

17 = {u e R"* : 7 j(u) < 0,7 = 1 ,... ,r} 

with r > m, where the functions yj are convex and of class C^. Further assumptions on the 
functions {yj,j = 1 ,... ,r} and {gi,i = 1 ,. .. ,p}, associated to the constraints, are imposed in 
(A4)-(A5) (see below). The input function u is assumed to belong to the set U of Lebesgue 
measurable functions from to Lf, i.e. u is a measurable function such that uit) G U for 

almost all t G [to, oo). 

x“(t) or denotes the solution of the differential equation ([T]) with input u G U and 

initial condition ([2]). 

Let us stress that the constraints O, called mixed constraints [Mil], depend both on the 
state and the control. We denote by g{x,u) the vector-valued function whose i-th component is 
gi{x,u). By g{x,u) -< 0 (resp. g{x,u) ■< 0) we mean gi{x,u) < 0 (resp. gi{x,u) < 0) for all i. By 
g{x, u) A 0 , we mean gi{x, u) = 0 for at least one i. 

We define the following sets: 


(5) 

( 6 ) 

(7) 

( 8 ) 


G = {x S R" : g{x, u) A 0} 


u^U 


Go = {x G G : min max gi{x, u) = 0} 

u£U 2 €{ 1 ,..., p } 

G- = {a: € R" : g{x, u) -< 0} 

ueu 

U{x) = {u gU : g{x,u) AO} Vx G G. 


Given a pair (x, u) G R" x U, we denote by I(x, u) the set of indices, possibly empty, corre¬ 
sponding to the “active” mixed constraints, namely: 

I(x,u) = {* 1 ,... = {i G {!,.. .,p} : g^{x,u) = 0} 

and by JJ(u) the set of indices, possibly empty, corresponding to the “active” input constraints: 

= {ii, ■ • ■ ,7^*2} = {j G { 1 , •■•?'} : Ijiu) = Oj- 

The integer si = #(I(x,it)) < p (resp. S 2 = #(!(«)) < r) is the number of elements of I(x,'u) 
(resp. of J(u)). Thus, si -I- S 2 represents the number of “active” constraints, among the p + r 
constraints, at (x,u). 

According to [16] a Lebesgue point, also denoted by L-point for notational convenience, for a 
given control u G Li is a time t G [to,c») such that u is continuous at t in the sense that there 
exists a bounded (possibly empty) subset Iq C [to, 00 ), of zero Lebesgue measure, which does not 
contain t, such that u{t) = lims_,.t_s^ 7 (, u(s). Since u is Lebesgue-measurable, by Lusin’s theorem, 
the Lebesgue measure of the complement, in [to,7’], for all finite T, of the set of Lebesgue points 
is equal to 0 . 

Note that if ui G and U 2 G lA^ and if t > to is given, the concatenated input v, defined by 

v{t) = \ ^ [^o,'r[ gg^i-jgfjgg y The concatenation operator relative to r is denoted 

} U 2 (t) it t > T 

by Nt-, i.e. v = ui Nt- U 2 . 

We further assume: 


(Al) / is an at least C^ vector field of R" for every u in an open subset Ui of R™ containing U, 
whose dependence with respect to u is also at least C^. 
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(A2) There exists a constant 0 < C < +oo such that the following inequality holds true: 

sup \x^f{x,u)\ < C{1 + ll^lp), for all x 

u^U 

(A3) The set f{x,U), called the vectogram in [TT], is convex for all x G M”. 

(A4) g is at least from R” x C/i to Moreover, the (row) vectors 

(9) |^(x,'u), : * G G J(m)| 

are linearly independent at every (x,u) G M" x Lf for which I(a:, u) or JI(m) is non emptyH 
We say, in this case, that the point x is regular with respect to u (see e.g. [TCI [TU] 1. 

(A5) For all i = 1,... ,p, the mapping u i— gi(x, u) is convex for all x G M". 

Given u Gti, we will say that an integral curve of equation (ED defined on [to; 2 ^] is regular if, 
and only if, at each L-point t of u, x'^{t) is regular in the afore mentioned sense w.r.t. m(<), and, in 
the opposite case, namely if t is a point of discontinuity of u, a;“(t) is regular in the afore mentioned 
sense w.r.t. ■u(t-) and u{t+), with u{t-) = lim^-^j m(t) and u{t+) = lim^x^j t^/j, u{t), Iq being 
a suitable zero-measure set of R. 

Since system ED is time-invariant, the initial time to may be taken as 0. When clear from the 
context, “Vt” or “for a.e t" will mean “Vt G [0, oo)” or “for a.e. t G [0,oo)”. Note that throughout 
this paper a.e. is understood with respect to the Lebesgue measure. 

3 The Admissible Set: Topological Properties 


Definition 3.1 (Admissible States) We will say that a state-space point x is admissible if there 
exists, at least, one input function v Gti, such that ED-(HD are satisfied for xq = x and u = v: 

(10) A= {x G G :3u GU, g{x^^’^\t),u{t)) <{),ior a.e. t}. 

According to the Markovian property of the system, any point of the integral curve, x^'"’^\t'), 
t' G [0, oo), is also an admissible point. 

The complement of A in G, namely A^ = G\A, is thus given by: 

(11) A^ = {x G G : Wu G U, 3i G {1, .■.,p}, 3t < -boo, L — point, s.t. gi{x^'^'^\t),u{f)) > 0}. 

From now on, all set topologies will be defined relative to G. We assume that both A and A^ 
contain at least one element to discard the trivial cases A = % and A*” = 0. 

We use the notations int(5') (resp. 01(5)) (resp.co(5')) for the interior (resp. the closure) (resp. 
the closed and convex hull) of a set S. 

As in |7], we also consider the family of sets At, called finite horizon admissible sets, dehned 
for all finite 0 <T < -boo by 

At = {x G G :3u GU, g{x'''^'^\t),u{t)') ^ 0,for a.e. t <T} 
as well as its complement A'f in G is given by: 

Ay = {x G G -.^u GU, 3i G {1, ...yp}, 3i <T, L — point s.t. gi[x^'^’^^t),u{t)) > 0}. 
Clearly, since A C At for all finite T, we have At ^ 0- 


^Note that this implies that si + S2 < m, with si = u)) and S2 = #(J(w)) 
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Proposition 3.1 Assume that (A1)-(A5) are valid. The set of finite horizon admissible states, 
At, is closed for all finite T. 

Proof. The proof follows the same lines as Proposition 4.1 of [7], up to small changes. We sketch 
it for the sake of completeness. 

Consider a sequence of initial states {a;fc}fegN in At converging to a; as A: tends to infinity. By 
definition of At, for every fc € N, there exists Uk G 14 such that the corresponding integral curve 
^{uk,xk) gatisfies g{x^'^’^'^^\t),Uk{t)) ^ 0 for a.e. t S [0,T]. According to Lemma lA^l there exists 
a uniformly converging subsequence, still denoted by to the absolutely continuous integral 

curve for some u Gl4. Moreover, we have {t),u{t)) ^ 0 for almost every t G [0,r], 

hence x G At, and the proposition is proven. ■ 

Corollary 3.1 Under the assumptions of Provosition HOI the set A is closed. 

Proof. See proof of Corollary 4.1 of [7]. ■ 


4 Boundary of the Admissible Set 

4.1 A Characterisation of A, its Complement and its Boundary 

Denoting by DAt (resp. dA) the boundary of At (resp. A), we know from Proposition 13.11 and 
Corollary 13.11 that OAt C At (resp. dA C A). Following [7], we focus on the properties and 
characterisation of these boundaries. 

We first prove the following result, where the notation ess. sup(g[Q_oo) h[f), with h : [0, oo) ^ R 
measurable, stands for the L°°{Q, oo)-norm of h. 

Proposition 4.1 Assume that (A1)-(A5) hold. We have the following equivalences: 

(i) X & A is equivalent to 

(12) miness.sup max gi{x^'^’^\t),u{t)) < 0 

tG[0,oo) 


(a) X G A^ is equivalent to 

(13) miness.sup max g^{x^'^’^\t),u{t)) > h 

tG[0,oo) 


(Hi) X G dA is equivalent to 

(14) miness.sup max gi{x^^'^\t),u{t)) = 0. 

te[0.oo) 

Proof. We first prove (i). If a; G A, by definition, there exists u GU such that g{x^'^'^^ [f), u{t)) ^ 0 
for almost all t > 0, and thus such that ess. sup^^jQ maxi=i_..._p ( 7 i(x(“’'^^(t), u(t)) < 0. We 
immediately get 

(15) inf ess. sup max gi{x^'^’^\t),u(t)) < 0. 

uGU tg[o,oo) *=L---,P 

Let us prove next that the infimum with respect to u is achieved by some u GU va. order to get 
m- To this aim, let us consider a minimising sequence G U, k G N, i.e. such that 

(16) lim ess.sup max gi{x^'^'°’^\t),Uk{t)) = ini ess.sup max gi{x^'^'^\f),u{t)). 

k^oo tg[o,oo) ueU tg[o,oo) i=l.---.P 
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According to Lemma lA.2l in Appendix lAl with Xk = x for every /c S N, one can extract a uniformly 
convergent subsequence on every compact interval [0,r] with T > 0, still denoted by 
whose limit is for some u G U. Moreover, one can build another subsequence, made of 

convex combinations of the {g{x^‘^'‘’^\uk)}, namely where the are 

all non negative real numbers such that W j ~ 1 alH = 1 ,... ,p and fc > 1, that pointwise 

converges to g{x^‘^’^\u) a.e. t G [0,T] for all T > 0. 

According to Egorov’s theorem [18], the pointwise convergence implies that, for almost every 
t G [0, T], all T > 0 and e > 0, there exists ko{t, T, e) G N such that, for every k > ko{t, T, e), 

k 

g,{x^^’^\t),u{t)) <'^aljgi{x^^^’^\t),Uj{t))+e. 

1=1 

Taking the maximum with respect to i G {1,... ,p} and the essential supremum w.r.t. t G [0, oo) 
on the right hand side, we get 

k 

gi{x^'^’^\t),u{t)) af ess. sup max gi{x^'^^’^\s),Uj{s)) + e VtG[0,r]. 

^ se[0.oo) *=i.-,p 

On the other hand, by the definition of the limit in (fTBl) . for every e > 0 there exists fci(e) G N 
such that for all j > ki {e) , we have 

ess.sup max gi{x^'^^’^\t),Uj{t)) < inf ess.sup max gi{x^^’^\t),u{t)) + e 
te[o.oo) j=i.---.p ueu tg[o,oo) j=i.---,p 


and thus 


k 

gi{x^'^’^''(t), u{t)) < 'y' afMnf ess.sup max gi{x^'^’^\t), u{t)) + 2e VtG[0,r]. 
^ ueu tg[o,oo) j=i.---,p 


Hence, using the fact that = 1, for all k > max(fco(t, T, e), fci(e)), we get 

< inf ess.sup max M(t)) + 2e a.e.tG[0,T], Vi = l,...,p. 

tG[0,oo) *=i.---.p 

However, since the latter inequality is valid for any t and T > 0 and it does not depend on k 
anymore, and since its right-hand side is independent of i, t and T, we have that the inequality holds 
if we maximize the left-hand side with respect to i G {1,... ,p} and take its essential supremum 
with respect to t G [0, oo). Thus, using the definition of the infimum w.r.t. u, we obtain that, for 
every e > 0 

ess. sup max gi{x^'^’^\t),u{t)) < inf ess. sup max gi{x^'^'^\t),u{t)) + 2£ 

tG[0,oo) i=l,---.P UGW tg[0.oo) i=l.---.P 

< ess.sup max gi(x^'^’^\t),u{t)) + 2e, 

ie[0.oo) *=i.---.p 

or, using also (da, that 

ess. sup max (t), it(t)) = inf ess. sup max «(!))< 0, 

te[0.oo) *=1.---.P ■>^£-U tg[0,oo) i=l.---.P 

which proves (ini)- 

Conversely, if (1121) holds, there exists an input u GU such that 

ess.sup max gi{x^'^'^\t),u{t)) < 0, 

tG[0,oo) *=i.---.p 
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which in turn implies that g{x^'^’^\t),u(t)) ^ 0 for almost all t > 0, or, in other words, x € A, 
which achieves the proof of (i). 

To prove (ii), we now assume that x S and prove p3l) . By definition of A^^, for all u € U, 
we have ess. sup^^jQ gi{x^'^’^\t)) > 0 and thus 

inf ess. sup max gi{x^'^’^\t), u{t)) > 0. 

"eW tg[o,oo) 

The same minimising sequence argument as in the proof of (i) shows that the minimum over u GU 
is achieved by some u € U and that 

miness.sup max gi{x^'^’^\t),u(t)) > 0. 

"eW tg[o,oo) 

But the inequality has to be strict since, if min ess. sup max gi{x^'^’^'^ (t),u{t)) = 0, it would imply, 

tG[0,oo) 

according to (i), that x € A which contradicts the assumption. Therefore, we have proven CSl). 

Conversely, if m holds, it is immediately seen that x is such that 

ess. sup max gi{x^'^’^\t), u{t)) > 0 for all u GU. The essential supremum with respect to t must 

te[o.oo) 

be reached at some i{u) < +oo since i{u) = +oo would imply that maxj=i_..._p gi{x^'^’^'> (t), u{t)) < 0 
for almost all t < +oo, and thus ess. sup max gi{x^'^’^\t),u(t)) <0. A fortiori, 

tG[0.oo) i=l,---.P 

miness.sup max gi{x^'^'^\t),u{t)) < 0, which contradicts ([13]). Thus, for all u G U, there 

“ew tefO.oo) i=l.---.P 

exists i{u) < +oo such that max gi{x^'^’^\t{u)), u(t{u))) > 0, and hence x G A*", which proves 

(ii)- 

To prove (iii), since A is closed, x G dA is equivalent to x G A and x G cl(A‘'), the closure 
of A^, which, by (i) and (ii), is equivalent to (fT^ and (I13p (the latter with a “>” symbol as a 
consequence of a; G cl(A‘')), which in turn is equivalent to (fl^ . ■ 

Remark 4.1 The same formulas hold true for At, A^ and dAr if one replaces the infinite time 
interval [0, oo) by [0,T]. 

4.2 Geometric Description of the Barrier 

As a consequence of (1141) . the boundary dA is made of points x such that there exists a. u GU for 
which at least one of the constraints is saturated for some L-point i, i.e. g{x^'^’^'> {t),u(t)) A 0. As 
in [7], let us define the set: 

[dA]_ =^AI^G- 

Definition 4.1 The set [9A]_ is called the barrier of the set A (see Corollary 14-11 and W- 

Proposition 4.2 Assume (Al) to (A5) hold. [9.4]_ is made of points x G G- for which there 
exists u GIA and an integral curve entirely contained in [9A]_ until it intersects Go, i.e. at 

a point z = x^'^’^\i), for some i, such that mmu^u gi{z,u) = 0. 

Proof. Let x G [9.4]_, therefore satisfying (IT4l) . In particular, there exists u GU and t > 0 such 
that 

miness.sup max gi{x'^’^{t),u{t)) = ess.sup max gi{x'^’^{t),u{t)) 

“SW tg[o,oo) i=l.---.P ie[0.oo) i=l.---.P 

= max gi{x^'^’^\t),u{i)) = 0 
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where u has been possibly modihed on a 0-measure set to satisfy the right-hand side equality. Then, 
choose t as the first time for which maxi=i,,,,_p = 0 and an arbitrary to G [ 0 ,t[. 

Setting v[t) = u{to -f t), since to < i, the point ^ = x^'^’^'>{to) satisfies maxj^i^,,,_p i^(0)) = 
Hto) ,u{to)) < 0, i.e. ^ € G-, and by a standard dynamic programming argu¬ 
ment (since x^''’^\t) = x^^'^'^{to -ft) for all t > 0 ), min ess. sup max gi{x^^'^\t),u{to +t)) = 0 . 

te[o.oo) 

It follows that ^ G and, therefore, the arc of integral curve between 0 and t starting from 

X G [9^]_ is entirely contained in [dA\_. 

We now prove that this integral curve intersects Gq. Since x G [9-4] _, there exists an open 
set O C K" such that x + eh & for all h G G and ||/i|| < H, with H arbitrarily small, and 
all e sufficiently small. Therefore, there exists t^^h such that maxj=i^,,,_p u(t)) < 0 

for all t < ts,h and maxj=i ,,,_p u{tg^h)) > 0- Taking an arbitrary a G]0,fE^/i[ 

and setting we indeed have G G_. Assume, by contradiction, that 

there exists u £ U such that maxi^i^...^p gi{x^'^’^^’'''> {t),u{t)) < 0 for all t £ [cr, cr -I- t[ for some 
sufficiently small r > 0 and ( = -I- r) G int(A), which indeed implies that a:*^“4e,h) (i) g 

G- for all t £ [cr, cr -I- r[. As a consequence of (HID and d, there exists v £ U such that 
ess. sup max gi{x^'"'^\T + a + t),v{T + a + t)) < 0. Setting v = u >ir u Mr+a- v, we easily 

te[o.oo) r=i.....p 

verify that ess. sup max gi{x^'"’^~^‘^’^\t),vit)) < 0 , which implies, again by (|T^ and (fT4)l . that 

te[ 0 ,oo) i=G....p 

X + eh £ int(-4), the whole integral curve remaining in G_, hence contradicting the fact 

that x + eh £ A^. We thus conclude that no integral curve starting in -4*” can penetrate the interior 
of A before leaving G_. Note that along the same lines and taking the limit as e, ||ft,|| —0, we 
prove the same result for [9-4] _ (see Corollary 14. II) . 

Note that x £ G_ is equivalent to int(17(a;)) ^ 0. Thus, because [9.4]_ C G_, we can conclude 
that X £ [9-4]_ implies x^'^’^\t) £ [9-4]_ for all t such that \nt{U{x^'^’^\t))) ^ 0. It results that 

i=sup{t £ [0,oo) : G [9A]_} = sup{f G [0,oo) : G(x(“'®)(t)) ^ %,x^^’^\t) £ [9A]_} 


Thus t satisfies 


min max gi{x^'^’^\t),v) 
v^U 


.max g,{x^^’^\^,u(j}) = 0 , x^^’^\T) £ cl([9-4]_) 


which proves that the arc of integral curve intersects Gq. ■ 

In the course of the proof of Proposition 14.21 we have proven the following result which is of 
interest by itself (semi-permeability): 

Corollary 4.1 Assume (Al) to (A5) hold. Then from any point on the boundary [9A]_, there 
cannot exist a trajectory penetrating the interior of A before leaving G_. 


4.3 Ultimate Tangentiality 

We now characterise the intersection of [9A]_ with Go at the point z defined in Proposition 14.21 
We define 

(17) 5 (x) = min max gi{x,u). 

u£U 2G{1,...,p} 

Comparing to ([ 6 ]) we immediately see that Go is the set of points x £ G such that g(x) = 0. We 
prove that g is locally Lipschitz, a simplified version of a result of J. Danskin [^: 

Lemma 4.1 The function g is locally Lipschitz, and thus absolutely continuous and almost every¬ 
where differentiable, on every open and bounded subset o/K”. 
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Proof. Consider the family of subsets of 0*=! ^(] “ oO;0])) defined by 

Oj = {{x,u) € n c\{g~^{]-oo,Q])) : ma.x gi{x,u)=gj{x,u)}, j = l,...,p. 

It is clear that Uj=i p^j = ni=i ~ oo,0])) and that we can extract a minimal 

subfamily of {Oj} still covering Hi^i p “OOj 0])), where every Oj has non-empty interior. 

In the sequel we only consider this subfamily. Given xi and X 2 in G_ arbitrarily close, there exists 
ii such that {xi,ui) £ Oi-^ with ui such that gi-^{xi,ui) = mhiueu 9 iiixi,u), and such that 
{x 2 ,ui) € Oi^. Thus, we get 

g{x 2 ) — g{xi) = umi max gi{x 2 ,u) — min max gi{xi,u) 
u^U 2G{1,...,p} u£U 


(18) 


< max gi{x 2 ,ui) — max gi{xi,ui) 

< max giix 2 ,ui) - g^^{xl,Ul) 
ie{i,...,p} 

< gi^{x2,ui) -5q(xi,ui) 


Thus, since g is continuously differentiable in x for all u, there exists a point such that 
gij^ix 2 ,ui) - = Dxgi^i^i,ui) (x 2 - xi). 

Similarly, there exists i 2 such that (x 2 ,U 2 ) £ Oi^ with gi 2 {x 2 ,U 2 ) = miiiueu 912 (^ 2 , u) and 
(a;i,M 2 ) £ Oi^. We get 


(19) 


9i2(.^2,u2) - 9i2{xi,U2) < g{x 2 ) - g{xi) 


Again, there exists a point ^2 such that gi 2 {x 2 ,U 2 ) — gi 2 (,Xi,U 2 ) = (^ 2 )^ 2 ) (x 2 — Xi). Com¬ 

bining (fTSl) and (ITO)) yields 

15 ( 3 ^ 2 ) - 9{xi)\ < C\\X2 - xill 

with C = sup(||Da: 5 ii (Cl: ui)||, ||Da: 5 i 2 (^ 2 : ^2 )||)■ It results that g is locally Lipschitz. The absolute 
continuity and almost everywhere differentiability follow from Rademacher’s theorem (see e.g. [21 
Theorem 3.1]. See also [3l[5]), which achieves to prove the lemma. ■ 

We summarise a few concepts from nonsmooth analysis [5] that will be used in the next propo¬ 
sition. Consider ft, : A —M, where A is a finite dimensional vector space, and ft is Lipschitz with 
Lipschitz constant K near a given point x G X. The generalised directional derivative of ft at a; in 
the direction v is defined as follows: 


( 20 ) 


hP{x]v) = limsup 

y—>^0+ 


Ky + tv) - h{y) 


t 


We also need to introduce the generalised gradient of ft at x, labeled dhix). It is well-known 
that in our setting, where we consider a Lipschitz function ft : R" —R, the generalised gradient is 
the compact and convex set: 


( 21 ) 


dh{x) = co{ lim Dh^(xt) : Xi ^ x,Xi ^ fliU ^ 2 } 


where Dh?"{x) denotes the transpose of the row vector Dh{x) at x, fli is a zero measure set where 
ft is nondifferentiable (recall that ft is differentiable almost everywhere), 172 is any zero-measure 
set and recall that 00(5”) denotes the closed and convex hull of an arbitrary set S. Equivalently, 
denoting by B^^x) the open ball of radius e centered at x, we have: 

dh{x) = n n CO [Dh^{Bg{x) \ 17)) 

£>0 meas(n)—0 
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The relationship between the generalised directional derivative and the generalised gradient is 
given by: 

(22) h^(x]v) = max 

iedhix) 

Proposition 4.3 Assume (Al) to (A5) hold. Consider x G andu GU as in Proposition^^ 

i.e. such that the integral curve x^'^'^\t) G [9.4]_ for all t in some time interval until it reaches 
Go at some finite time t >0. Then, the point z = x^'^’^\t) G cl([9.4]_) fl Go, satisfies 

(23) 0 = max e'^f{z,u{t)) = niin inax £^/(z, v) = max min F'^fiz,v). 

^Gdg{z) vGU{z) ^Gdg{z) ^&dg{z) veU{z) 

Moreover, if the function g is differentiable at the point z, then condition (1231) reduces to the smooth 
counterpart: 

(24) 0 = Lfg{z,u{t)) = mm Lfg{z,u) 

uGU{z) 


where Lfg{x,u) = Dg{x)f{x,u) is the Lie derivative of g along the vector field f at {x,u). 

Proof. Let xo G [dA\_, then there exists a u G U such that g{x'^'^’^°\t)) < 0 until 
intersects Go at some i. As in the proof of Proposition 14.21 we consider an open set O C K" such 
that xo+sh G A*" for all h G O and ||h|| < H, with H arbitrarily small, and all e sufficiently small. 

Introduce a needle perturbation of u, labeled UK,e, at some Lebesgue point r of u before 
intersects Go, in the spirit of [7], i.e. a variation of u, parameterized by the vector 

«: = (u,T,/) e -le)) x [0,r] x [0,L] 


with bounded T, L, of the form 


(25) 


= u 


IX(r-le) V Mr U = 


V on [T — le,T[ 

u elsewhere on [0, T] 


where v stands for the constant control equal to u € U {x^'^'^°\t)) for all t G [t — 1£,t\. Remark 
that, by definition of G_ and U{x), since x^^’^°\t) G G_ for all t < i, we have u{t) G U{x^^’^°\t)) 
for alH < t and thus (0) ^ 0 for all t < t. 

Because xo + eh G A^, < oo at which crosses Go, see Proposition 

14.21 As a result of the uniform convergence of 2 ;(“'=,e. 2 :o+e?») j.O,xo)^ there exists a t > i, s.t. 

as £ —>■ 0 and, according to the continuity of g, we have 

lim5(x(“--"°+^fo(G,.,,,)) = 0 = g(x(“’"«)(t)). 

£->•0 


Because = 0 and g{x^'^'^°\te^K,h)) < 0 (recall that g{x^'^'^°\ts^K.,h)) < 

g{x^'^’^°'> {te,K,h),u{ts^K.,h)) < 0 since the pair {x^'^’^°\t),u{t)) satishes the constraints for all t), we 
have that 

5(x(“--"°+^fo(G...?.)) -g(:r(“’"o)(t,_,,^)) > 0. 

Recall from [16] as well as [7] that 




_ ^{u,xo) 


(te.K.h) + £w{te,K,h, K, h) + 0{e^) 


where 


w{t, K, h) A ^^{t, Q)h + l^^ft, r) (/(x("’"“)(t), v) - /(x(“’"«)(r), u(t))) , 
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(t, s) being the solution to the variational equation at time t starting from time s (see equation 
(l48l) in Appendix I bI). t being any Lebesgue point of the control u, with v G U and where 
we have denoted by O(e^) a continuous function of defined in a small open interval containing 

0 and such that lim£_>.o = lime_j>o 0(e’') = 0 for all 0 < r < fc — 1, A:, r € N. 

Since g is almost everywhere differentiable, we have: 


(26) 




= Dg{x^'^'^°\ts^K.,h))-w{te,K.,h, K, h)) + 0{e) > 0 


for every v G C/(x^“’““^(r)) and almost every e and h. 

If we take any accumulation point of the right-hand side of (l26l) as e and ||/i|| tend to zero, 
according to (uni) and (1211), we get, after division by 1: 


(27) ^^$«(t,r)(/(:r(“’"“)(T),u)-/(x(“’"°)(T),u(r))) >0 G d~gix^^’-°\t)) 

Assume for a moment that we can replace v in (1271) by a continuous family with respect to t 
such that lim^^fU,- = v. This result is proven in Lemma 14.21 below. Thus, taking the limit as r 
tends to i in (E71) , we get 

(28) {f{z,v) - f{z,u{t)) >0, y^Gdg{z), \IvGU{z) 
where z = x^“’“°)(t). Therefore, 

(29) max f^f{z,u{i)) = min max I^f{z,v). 

^Gdg{z) v^U{z) i^dg{z) 


Since the mapping ^ f{z,v) is linear on the compact and convex set dg{z) and the mapping 
V I—>■ f^/(-z,u) is convex and continuous on the compact set U{z) which is convex by (A.5), it 
results from the minimax theorem of Von Neumann (see e.g. [1]) that 


(30) 


min max ^ f(z,v) 

vGU{z) ^€dg{z) 


max min 

^edg{z) v£U{z) 




If t is not an L-point, it suffices to modify u on the 0-measure set {t} by replacing u{t) by its 
left limit u(t-) in the latter expression. 

We will now show that this expression is equal to 0. On the one hand, because g is locally 
Lipschitz, Dg exists almost everywhere and the mapping 1 1 —>■ g{x^'^’^°\t)) is nondecreasing on some 
small interval {i—g,i\ with 77 > 0 sufficiently small, and we have (t))./(x^“’“°)(t), it(f)) > 

0 where Dg exists. Therefore we conclude that 

(31) g°(z;/(z,u(t_))) > 0. 


On the other hand, by definition, we have: 


£ 


(te,.. 7 .)) -s(x(“’"«)(te,«,/*))] 


rg(x(“’"“)(te,K.7.))-Kx(“’"°)(t))1 

£ 


£ 


Thus, since the first bracketed term of the right-hand side has been proven to be > 0, we immedi¬ 
ately get 


lim sup 

£—^0-f- 


g(a:(“-"°)(f))-g(a7("-"»)(t,,^,,,)) 


e—J-O-f 


e 


e 
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But, since 


lim sup 

e—J-O-f- 


£ 


= -~g°{z-,f{z,u{i.))) 


we conclude that —g°{z;f{z,u{t-))) > 0. Comparing to (IHTl) . we get g°{z; f{z,u{t_))) = 0, or 
according to ( 1 ^ : 

0 = rnaf e^fiz,u{t)) 

i&dg{z) 

which, together with (0^1) and (1501) . proves (1^ . 

If g is differentiable at z, we can apply exactly the same argument as before up until equation 
(EH). Thus, letting ||/i|| —>• 0 and dividing by we get: 




If £ now tends to zero we get 


We again assume that t is an L-point for the control u, and construct the same continuous mapping 
T M- Ut- as before, such that lim^^fu,- = u, for an arbitrary v € U{z) to get: 

Dgiz)f{z, v) > Dg{z)f{z, u{t)), Vv € U{z) 


or, using the Lie derivative notation: 

Lfg{z,u{t))= min Lfg{z,v). 

vGU(z) 

Interpreting Lfg{z,u{t)) as the time derivative of t g{x^'^’^°^(t)) and remarking that the 
latter mapping is non decreasing on an interval ]i — g,t\, for some rj > 0 small enough, we indeed 
deduce that Lfg{z,u(t)) > 0. The same mapping being non increasing on the interval [i,i + g'[, 
we have Lfg(z^u{t)) < 0, which finally proves that Lfg(z,u(t)) =0. If t is not an L-point of u, 
the same modification of u at t, as in the nonsmooth case, may be applied, which achieves to prove 
the proposition. ■ 


Lemma 4.2 Under the assumptions of Proposition for all v S U{z) with z = x^'^’^°\t), there 

exists a continuous mapping r i—> Ut from [t — 77 , t[ to U, with 77 > 0 small enough, such that 
Vt G U{x^'^'^°\t)) for all t € [t — 77 ,f[ and lim Ut = v. 

T/'t 


Proof. Recall that the condition Vr G C/(a;*^“’““^(r)) is equivalent to g{x^'^’^°\T),Vr) 0 for all 

r S [t — 77 , t[ and, since 2 : = x^'^’^°'>{t) G Gq, v G U{z) is such that g{z, v) = 0. We construct such a 
Vt as follows. 

Since, by assumption, #I(z,u(t_)) = Si and ffj{u{t_)) = S 2 , with max(si,S 2 ) > 0, consider 
the equation 

/ g^,{x,u) \ 


r(a;, u) = 


9^.,{x,u) 
In {u) 


= 0 . 


V / 

According to assumption (A4) and the implicit function theorem, there exists a continuously 
differentiable mapping: 







BARRIERS IN NONLINEAR CONTROL SYSTEMS WITH MIXED CONSTRAINTS 


13 


M = (ui,..., zisi+sj : K" X ^ 

defined in a neighbourhood of the point (2, , Vm), labelled J\f, such that 

, Um)j '*^si+S2 + lj ■ • ■ j ’^m) — V 

and 

r(^, u{x^ 'as^-|_s2 + l; ■ - ■ 1 Um): +S 2 + I : ■ ■ • : ^m) — 0 ^(^j^si+S2 + lj-*-; ^m) ^ Af. 

Then we define 

Vt = m(x^"’'^”^(t),Usi+s2+1, . . .,Vm) Vt S [t - 7?, t[ 

with Tj small enough such that Usi+sa+i, • ■ ■, I'm) remains in Af in the whole interval 

[t — 77, i[. Therefore, we have r(a;^“’^”)(r), Vr) = 0 for all r G [t — 77, i[. Moreover, since Vr so defined 
is clearly a continuous function of r, and since, by assumption (A. 4 ), 77 may be possibly decreased 
in order that 


g*(a;(“’““^(T),7;T-, 77*1+82+1, ••.,Um) <0 Vr G [t-77,t[, 


Vi ^ 1 ( 2 :,7t(t_)) 


and 

77^, 77*1+82+1, ■■•,77m) <0 Vr G [i-tiJl 
we have, as required, Vr G (r)) and limi-yi(77T = v. 


Vj ^ J(7i(t_)) 


5 The Barrier Equation 


We next present the main result of the paper. Theorem 15.11 which gives necessary conditions 
satisfied by an integral curve running along the barrier. The proof of the theorem utilises the 
maximum principle for problems with mixed constraints stated in terms of reachable sets where 
the extremal curves are those whose endpoints at each time t belong to the boundary of the 
reachable set at the same instant of time. See the Appendix |B] for more details. 

Theorem 5.1 Under the assumptions of Proposition U^ consider an integral curve x“ on [i9Vl]_n 
cl(int(A)) and assume that the control function u is piecewise continuous. Then u and a;“ satisfy 
the following necessary conditions. 

There exists a non-zero absolutely continuous adjoint A“ and piecewise continuous multipliers 
ptf >0, i = 1,... ,p, such that: 

(32) X^t) = - f^ia:^t),u{t))y 

^ i—1 

with the “complementary slackness condition” 

(33) p'“{t)gi{x^{t),u{t)) =0, i = l,...,p 
and final conditions 

(34) A“(t)’^ G arg max f.f{z,u(t)) 

iedgiz) 

where z = x“(T) with t such that z G Gq, i.e. min„gj/max^^i ^ 77 ^( 2 , 7 t) = 0, dg{z) being the 
generalised gradient of g defined by (HZl) at z. 
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Moreover, at almost every t, the Hamiltonian, H{X^{t),x^{t),u) = (A“(l)) f{x^{t),u), is 
minimised over the set U{x^{t)) and equal to zero: 


( 35 ) u£U{x'^{t)) ueu 


(A“(l))^ /(x“(l), u)+j2 l^'iit)9iix^(t),u) 


2 = 1 


= A“(l)'^/(a;“(l),M(l)) = 0 

Remark 5.1 To compute (1351) the following necessary conditions are useful: 


(36) 


H{\^{t),x^{t),u{t)) = Q 

lJ-iit)9zix'^it),uit)) = 0, i = 

, it)l3 (w W) =0) ^'“(i) > 0 j = 1,..., r. 


Before proving Theorem 15.II we need to introduce the following definition: 


Definition 5.1 The constrained reachable set at time t from initial condition x is given by: 

Rt{x) = {cc € K." : 3u S s.t. x = x^'^'^\t), g{x^'^'^\s), u{s)) ^ 0 for a.e. s <t} 

Lemma 5.1 Let x G [dA\_ ncl(int(^)) and u €ll as in Proposition^^ i.e. such that x^'^'^\t) G 
[dA\_ for all t G [0,t[ where i is the time such that g{x^'^’^'> (t)) = 0. Then, x^'^’^\t) G dRt{x) for 
all 0 <t < t. 


Proof. We first prove that Rt{x) C cl(Al*') for all 0 < t < t. Assume by contradiction that for 
some 0 < t < t we have Rt{x) H int(Al) ^ 0. Then 3u £U such that x^'^’^\t) G int(Al) for some 
0 < t < t, which contradicts the fact that x G [dA\_ by Corollarv l4.ll hence Rt{x) C 01 (^ 1 *"). 

By complementarity int(Al) C Rt{x)^, and thus cl(int(Al)) C cl(i?t(a;)^). Thus, assume that 
X G [i9aI]_ ncl(int(Al)) and that there exists u G 14 as in Proposition [421 Then it can be shown as 
in the proof of Corollary [TT] that there exists a sequence {ifclfcgN, with Xk G int(Al), and a sequence 
{uk}keN, Uk G 14, such that every integral curve lies in int(Al) and the sequence 

converges uniformly to on every compact interval [0,r]. We therefore immediately deduce 

that x^'^'^\t) G [dA\_ (~1 cl(int(Al)) for alH < t and hence that x^'^'^\t) G cl(i?i(a;)‘'). But because 
x{u,x)f^t) G Rt{x), and since dRt{x) = Rt{x) nc\{Rt{x)^), we conclude that x^'^’^'>{t) G dRt{x). ■ 
Proof. [Proof of Theorem 15. 1| 

By Lemma [5.11 we know that x^'^'^\t) G dRt{x) for all 0 < t < f. Therefore, according to 
Theorem lB.il we know that u must satisfy (j56|l . Then, setting A“ = — 77 “ we get (1^ with (IM)) 
and that the resulting dualised Hamiltonian 'H{x,u, X, fT) = 'H{x,u, —p, fT), defined by (1551) . now 
must be minimised. Now taking the final conditions for A“ as in Proposition 14.31 namely (l23l) . we 
immediately deduce that at time i the minimised Hamiltonian must be zero, and thus the constant 
of (1561) is equal to zero. Finally, according to the complementary slackness condition, (iMl) . the 
minimisation of H becomes equivalent to (1351) which achieves the proof of the theorem. ■ 


Remark 5.2 If g is differentiable at the point z, condition (1341) reduces to its smooth counterpart, 
i.e., A“(t)'^ = Dg{z) 

Remark 5.3 The assumption that G [clAl]_ (~lcl(int(Al)) means that we possibly miss isolated 

trajectories which are in Al\ cl(int(A)). The existence and computation of such trajectories, if they 
exist, are open questions. 
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6 Examples 

6.1 Constrained Spring 1 


Consider the following constrained mass-spring-damper model: 


Xi 

X2 



0 

1 


U, | m | < 1 , X2 — u < 0 


where Xi is the mass’s displacement. The spring stiffness is here equal to 2 for a mass equal to 1 
and the friction coefficient is equal to 2. u is the force applied to the mass. 

We identify g{x,u) = X 2 — u, U = [—1,1] and g{x) = X 2 — 1- We also identify the following 
sets: G = {x & : X 2 <I}t Gq = {x & G : X 2 = 1} and Lf[x) = {u & U ■. X 2 <u <T\. Note that 

li z = ( 21 , 2 : 2 ) G Go, i.e. 22 = 1, then U{z) is the singleton U{z) = {1}. 

We have dg{z) = {(0,1)^} = Dg{zY' (which means that g is differentiable everywhere) and 
the ultimate tangentiality condition reads: 

min Dg{z)^f{z,u) = 0 

u^U{z) 


which gives 

min — 22 i — 222 + u = —2zi — 2 + 1 = 0 

uGU(z) 

Thus 2 = (—i, 1). 

Let us now compute X{i). From (IMl) . which here reduces to (I24L we get that A(t) = Dg{z) = 

( 0 , 1 ). 

We now construct the barrier by integrating backwards from 2 and X{i). From the minimisation 
of the Hamiltonian, H{x, A, u) = Aia ;2 + A 2 (—2a:i — 2 a :2 + u), condition (I35L we find that the control 
u associated with the barrier is given by 


min Aia ;2 + A 2 (— 2 xi — 2 x 2 + u) = 0 

X2‘^U<1 


which gives: 

if X2{t) < 0 

u{t) = 1 

if X2{t) > 0 

X2 if X2 £] — 1,1] 

— 1 if X 2 S] — 00 , — 1 ] 

if X2{t) = 0 

u{t) = arbitrary 

We note from condition (EH) that if the constraint is active (i.e. g{x,u) = 0 ), the costate 
differential equation is given by 


u{t) = 


ax ax 


and is otherwise (when g(x,u) < 0 ) given by 
(37) 


df'^ - 

A“ = -4- A“ = 
ax 


0 2 

-1 2 


0 2 

-1 2 


A“-m“ 


A“. 


0 


Recall that A 2 (t) > 0 and X 2 (t) > 0. Therefore, because A and x are continuous, u{t) = X 2 {t) 
over an interval before t. We can show that u{t) ^ 1 over this interval: if X 2 = 1 and u = I over 
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an interval before i, then we get i 2 = —2xi — 2 + l = 0ora;i = —^ which implies ii = 0 for all 
t €\t — r],t\, 77 > 0. However, we would also have xi = 1 over t €]t — 77,?], which contradicts the 
fact that xi = 0 over this interval. 

Therefore, only the constraint g is active over an interval before t, and by (I36|) . we obtain g, 
over this interval: 


”5-^ Mo” — A 2 — /i — 0 

ou ou 


and thus A 2 = g. In addition the adjoint satisfies: 


(38) 


A = 



Vt G]t — 77 , t] 


At some point in time before t, let us label this point t, we have A 2 (i) = 0 and it can be verified 
that, at this time, X 2 {i) = 0 and Ai(t) < 0. Let us prove that A 2 is negative on the interval [0,t|. 
If A 2 vanishes at some point in time, since we have 

A 2 = 2Ai = 0 

then A = 0 which contradicts our assertion. We conclude that over [0,t], A 2 is either everywhere 
positive or everywhere negative. 

If over this interval before t A 2 > 0, then the co-state dynamics are as before, and A 2 < 0 which 
is equivalent to — Ai -I- A 2 < 0, but this contradicts the fact that Ai(t) < 0. We can conclude that 
A 2 is negative before i, and that u = 1 over this period. The costate dynamics are then given by 
dSIl). The sign of A 2 then remains negative until the trajectory intersects Go again. The barrier is 
shown in Figure [63] 


Go 



Figure I: Admissible set of the constrained spring from examDle l6.il 


Remark 6.1 Note that Assumption (A4) does not hold true at the final point z since there are 
two active constraints for only one control. However, since this condition is violated only at this 
point, we may conclude by continuity that condition (|M)) still holds. 






BARRIERS IN NONLINEAR CONTROL SYSTEMS WITH MIXED CONSTRAINTS 


17 


6.2 Constrained Spring 2 


Consider the same mass-spring-damper system with the same constants as in the previous 
example, but with a richer constraint: 


(39) 


Xi 

X2 


0 1 
-2 -2 



0 \ 

1 


. ^2 ; V 

1 J 

'0 = {x : X 2 

= 0 U X2 


< 1 , X2ix2 — u) <0 


and from (j35p and (j34p we identify, in same manner as in the previous example, two points of 
ultimate tangentiality, namely z = (—^,1) along with X{i) = (0,1), and z = (t,—1) along with 
X{i) = (0, —1). We defer the treatment of the xi axis, which is also in Go, to the discussion below. 

From the minimisation of the Hamiltonian, which is the same as in the previous example, we 
find the control u: 


if X2{t) < 0 


if X2{t) > 0 


if A 2 (t) = 0 


l{t) = 


l{t) = 


1 

X2 

X2 

-1 


if 

if 

if 

if 


X2 €] 0 , 1 ] 

X2 e] - i,o[ 

X2 e]o, 1] 

X2 e] - 1 , o[ 


u(t) = arbitrary 

If we now integrate backwards from the points (—^, 1) and (A, —1) with the control u{t) we 
obtain the barrier as in Figure It turns out that along both curves u(t) = X 2 (t). 



Xi 


Figure 2: Admissible set of the constrained spring from example 16.21 

Let us now turn to the xi axis, where g = x 2 — \x 2 \ is non differentiable. For any z on the xi 
axis, we have U{z) = [—1,1] and dg{z) = co ((0, —1)^, (0,1)^) = {0} x [—1,1] and we must have: 

(40) min max £.f(z,u) = 0= min max £2{—‘2xi+u) 

«e[-i.i]?eag(s) «€[-!,1] 52 G[-i,i] 

For each —i < Zi < 5 equation (HOI) has a solution given by ^ = ( 0 ,sign(— 2 zi -|- u)) from which 
we deduce that u = 2zi. However, one can directly verify that the integral curves of (l3^ with 
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endpoints in the set [—^, 5 ] x { 0 } with the control u = X 2 all correspond to admissible curves (in¬ 
tegrated backwards) and therefore do not belong to the barrier, but that they make the constraint 
equal to 0 for it = X 2 for all x G x {0} and for all t. This attests that our 

conditions are only necessary and far from being sufficient. 

Remark 6.2 Note that, as in Examvle \6.1\. Assumption (A4) does not hold true at the final points 
z G Gq since there are two active constraints for only one control. Again, we conclude by continuity 
that condition (j34l) still holds. 

7 Conclusion 

In this paper we have extended the work on admissible sets and barriers, introduced in [7], to the 
case of mixed constraints. In particular, we have shown that the properties of the barrier in the 
mixed constraint setting prolong those in the pure state constraint setting, with some significant 
differences concerning its intersection with the set given by Gq = {x : min„g[/ maxj=i^...^p gi{x, u) = 
0 }, intersection that occurs tangentially in a generalised sense. 

We also had to adapt the minimum-like principle, that allows the barrier’s construction, as in 
Theorem 15.11 a form of the Pontryagin maximum principle, presented in Appendix IB. 21 in terms 
of the boundary of the reachable set, was needed. However, the result in this form is available only 
for control functions that are assumed to be piecewise continuous. The possibility of relaxing this 
assumption to merely measurable controls is an open question, and will be the subject of future 
works. 

Proving Theorem lB.il required the introduction of the regularity assumption (A4) to guarantee 
the existence of needle perturbations that satisfy the constraints, even when some of them are 
active. This assumption is also used in the proof of the ultimate tangentiality condition (1231) . 
However, assumption (A4) may appear to be too strict, especially on the set Go, since on Go u 
belongs to the boundary of U, thus adding at least one new constraint to the previous ones, and 
leading to a Jacobian whose lines are no more independent. However, it might be possible to 
avoid evaluating this rank on Go by a continuity argument. This point will be addressed in future 
research. 


A Compactness of solutions 

We slightly extend the compactness results proven in [71 Appendix A] to the mixed constraint 
context. We recall without proof, from [7], the following lemma and its corollary: 

Lemma A.l If assumptions (Al) and (A2) of Section\^hold true, equation {!]) admits a unique 
absolutely continuous integral curve over [to,-|-oo) for every u Gti and every bounded initial con¬ 
dition xq, which remains bounded for all finite t > to, 

(41) ||a:(t)|| < ((1 + lixof - if ^ Kit). 

Moreover, we have 

(42) ||a:(t) — a:(s)|| < G|t — s| 
for all t,s G [to, T] and all T > to, where 

(43) G= sup ||/(a:, m)|| <-|-oo. 

\M<K(T),u^U 
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Corollary A.l Let us denote by X{xq) the set of integral curves issued from an arbitrary xq, 
||xo|| < oo, and satisfying {!)), Qj, 

If assumptions (Al) and fASj of Section hold true, X{xq) is a subset o/C°([0, oo),R"), 
the space of continuous functions from [ 0 , oo) to R", and is relatively compact with respect to the 
topology of uniform convergence on T], R") for all finite T > 0. In other words, from any 

sequence of integral curves in X{xo), one can extract a subsequence whose convergence is uniform 
on every interval [0,r], with T > 0 and finite, and whose limit belongs to C'*^([0,oo),R”). 


We now adapt the proof of [71 Lemma A.2, Appendix A]. Since we strictly follow the same 
lines, only its modifications are presented. 


Lemma A.2 Assume that (Al), (A2) and (AS) of Section\^ hold. Given a compact set Xq of 
R", the set X = UxoGA’o is compact with respect to the topology of uniform convergence 

on C'’([0,T],R") for all T > 0, namely from every sequence C X one can extract 

a uniformly convergent subsequence on every finite interval [0,r], whose limit ^ is an absolutely 
continuous integral curve on [0, oo), belonging to X . In other words, there exists x G Xq and u & hl 
such that f{t) = x^'^'^\t) for almost all t > 0. 

Moreover, if the sequence Mfe)}fegN satisfies the constraint g{x^'^’°'^'‘i{t),Uk{t)) ^ 0 

for all k and almost all t, then the limit also does: g{x^'^'^\t),u{f)) ^ 0 for almost all t. 

Proof. Since Xq is compact, it is immediate to extend inequalities (1411) and (HTl) to integral curves 
with arbitrary Xq € Xq by taking, in the right-hand side of 62 ), the supremum over all Xq € Xq. 
Thus, by the same argument as in the proof of Corollary lA.ll using Ascoli-Arzela’s theorem, we 
conclude that X is relatively compact with respect to the topology of uniform convergence on 
C'’([0,r],R"), for all T > 0. The proof that, from every sequence C X, one can 

extract a uniformly convergent subsequence on every finite interval [0,T] whose limit f belongs to 
X is done exactly as in [7]. 

Accordingly, the sequence of functions {t g{x^'^^’^’’\t),Ukit)) : /c S N} is bounded in 
L^([0,T],R"') for every finite T, which implies that this sequence contains at least a weakly con¬ 
vergent subsequence (still denoted by g{x^'^'‘’^'‘\uk)). We denote by g its weak limit, independent 
of T as above. 

Recall from |7] that we denote x = limfe_,.oo and Fk{t) = f{x^'^’‘’^'‘\t),Ukit)). By Mazur’s 
Theorem (see e.g. m Chapter V, §1, Theorem 2, p. 120]), for every k, there exists a sequence 
{a (,..., of non negative real numbers, with such that the sequence 


is strongly convergent to 


F 

9 


in every L^([0,T],R”) for all finite T. Note that this property a 


fortiori holds true if we replace the sequence Fi by any subsequence 
by selecting a subsequence of indices ij such that, given e > 0 , 


F, 


constructed 


sup (||/(a;K’"’b)(i),uq(t)) -/(^(t),uq.(t))|| -f \\g{x^'^‘i’^'^\t),Uy{t)) - g{£,{t),Ui 
te[o,T] ^ 


< e 2 


-j 


for each j, which is indeed possible thanks to the uniform convergence of to f and the 

continuity of / and g. Note also that the limit f ^ j remains the same (for convenience of 
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notation, we keep the same symbols for the a^’s, but we remark that these coefficients have to be 
adapted relative to the new subsequence). 

We therefore deduce, following [7], that F{t) belongs almost everywhere to the closed con¬ 
vex hull of which is contained in f{^{t),Lf) according to (A3) and, with an 

obvious adaptation, that g{t) G g{^{t),U) for almost all t. We immediately conclude that if 
g(^^Ok,Xk)Uk{t)) 0 for all k and almost every t, it is the same for any convex combination and 
therefore g{t) ^ 0 for almost all t. 

Finally, again according to (A3) and (A5), there exists, by the measurable selection theorem 
[ 2 ] , u S Zi such that 


f fm.m 



a.e. t G [ 0 , oo). 


Thus, we conclude that ^ satisfies ^ = f{^,u) almost everywhere, with ^{0) = x G Xq. By the 
uniqueness of integral curves of (Cl), we conclude that ^(t) = a;(“’^)(t) almost everywhere and, thus, 
that ^ G X. Accordingly, we indeed have g{t) = g{^{t), ult)) = g{x^'^’^\t),u{t)) ^ 0 a.e. t G [0, oo), 
which achieves to prove the lemma. ■ 


B Maximum principle for problems with mixed constraints 


In this appendix we sketch a version of the maximum principle for problems with mixed con¬ 
straints describing the extremal curves as those whose endpoints at each time t belong to the 
boundary of the reachable set at the same instant of time. This form is useful to prove Theorem 
15.11 The proof draws content from |12| . where the principle is proved in the particular case of 
constraints on the control, and |16j . where the principle is proved in the context of optimising a 
cost function for systems with both constraints on the control and the state, but which are not 
mixed, though a remark indicating the possibility of its extension to mixed constraints is given in 
[TBl Chapter VI, §35]. See also [TUI Chapter 7] for a proof in the framework of the Calculus of 
Variations. For a survey on the maximum principles with mixed constraints, the reader may refer 
to [ 8 ]. 

In our treatment we will introduce the suitable perturbations to regular trajectories, similar to 
m, that are needed to generate the so-called perturbation cone, the latter being crucial to obtain 
the necessary conditions of the maximum principle. Throughout the analysis we assume that the 
extremal control is piecewise continuous as in the above cited references. 

B.l Control perturbations 

Consider an integral curve associated with the piecewise continuous control u, initiating 

from the point Xg- Let Tfc, k = 0,..., K, with tq = 0, be a collection of points of continuity of 
u such that — elk is also a point of continuity with Zfc > 0 for all e small enough. Assume 
that g{x^'^'^°\t),u(t)) ^ 0 for a.e. t G [rfc_i,Tfe[. We will perturb the control over the interval 
Ik = [Tk —elk, Tk[ and extend both the control and the integral curve between and Tk+i —elk+i in 
order to satisfy the constraints. This will be done by first making a subdivision a^, q = 1,... ,dk, 
namely Tfc = < ■ ■ ■ < < ■ ■ ■ < = Tk+i — elk+i, assumed to contain all discontinuities of 

u on the interval [Tk,Tk+i — elk+i[ and adapting u and its corresponding integral curve on each 
subinterval [crj, [, g = 1 ,..., dfc — 1 , using the implicit function theorem. 

At fc = 1, if Si = #I(a;(“’^'’)(Ti),it(ri)) = 0 and S 2 = #J('S('ri)) = 0 (we do not pass on the 
indexing of si and S 2 with respect to k and q to avoid too cumbersome notations), we introduce the 
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classical needle perturbation: ui^ = u defined on the interval with arbitrarily 

chosen in U— eh)) and constant over [n — eh,Ti[- We denote by 

Otherwise, by remarking that Si and S 2 are such that max(si, S 2 ) > Oj according to (A4), define 
the function 

/ g^^ {x, u) \ 


ri^{x,u) = 


9^.:^{x,u) 
Ml (u) 


\ / 


and consider the solution ui^ of T{x, u) = 0 , defined from a neighbourhood of R" xR"* (si+s 2 ) 

toRsi+s 2 ^ Thus, 


h/i (^5 (^5 +S2 + 15 • ■ • 7 ^m) j ^si + S2 + l ^m) 0 ^(^5^si+S2 + l5’'*T ) GAT,, 


and we can define the integral curve xi^ by: 


(44) X = f{x,Ul^{x,Vs^+s2 + l,---Mm),Vs^+s2 + l,---,Vm) 

starting from — eh), with (z;sj+s 2 +ij • ■ • i ^m) arbitrary in the projection of Afi^ on 

R™“('*i+®2)^ and such that u/i(t) = ui^{xi^ (t), ?;si+s 2 +i: ■ ■ ■ ,Vm) G U{xi^ (t)) for all t G [n —eh,Ti[. 
In this case we denote = xi-^ (ti ). 

We now consider the interval [ri,T 2 — eh]- If si = #I(a;(“’2^”^(ri), u ( ti )) = 0 and S 2 = 
#J(u(ri)) = 0 then u is kept the same on and we denote by m/j ^ u and 

^ = a;("’^^A(cr 2 — al). Otherwise, since si = (ti), u{ti)) and S2 = #J(h(Ti)) are such 

that max(si,S 2 ) > 0, according to (A4), define the function 


/ gi^ix.u) \ 


T/i i(a;,u) = 


9i.,{x,u) 

Ml ir) 


\ ) 


and consider the solution uj^ ^ of T/j^ i(a;,M) = 0, defined from a neighbourhood AZ/j ^ of R" x 
]^m-(si+S 2 ) R®i+® 2 . Thus, 


Tiii[x,uii 1 (^5 '^Si+S2 + 1; ■ • ■ 7 '^m); '^Si+S2 + 15 • ■ • ; ^m) — 0 V(x, 7X^2+52 + 17 ■ • ■ 7 l^m) G A //2 2 

and we can define the integral curve Xi-^ 2 by: 

( 45 ) X = /(x, ZX/2^2 (^7 1 ^ 52+52 + 1 7 ■ ■ • 7 '^m), l^S 2 +S 2 +l 7 ' ■ • 7 l^m) 

starting from at time ri and assume that the interval \ij\,(j\[ is small enough such that its 
solution remains in A//2 2 • 

We iteratively apply the same construction for all q = 2 ,... ,di and thus obtain the perturbed 
a;/2,, and u/2 ^ in each interval [cr^, [, g = 1,..., dfc — 1, satisfying the constraints. 

Then finally, for fc > 1, assuming that xj^, _2^_2 and ui^. j^_2 have been obtained, we construct 
^ik+i dfc+2-1 ^ik+i d^^i-i by replacing ri in the above algorithm by to finally get the complete 
perturbed trajectory. 

According to HU Chapter VI, § 34 ] we introduce the following notations: the perturbation 
parameters denoted by tt belong to the convex cone co{(Tfc, lk,Vk,e) : k = I,..., AT} and we note 
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XTr{t) = xii,^{t) previously defined with the vector of perturbation parameters tt if t £ [o'giO'g+i[- 
Then, for a given vector of perturbation parameters tt = {ti, ..., , akh, vi, ■ ■ ■, Vk}^ 

with afe > 0 and = 1, we have 


(46) XTrit) = + eSx{t) + O(e^) 

with 

K 

(47) Sx{t) = ^ ak^'^{t, Tk) 

k^l 

and <i>“ the transition matrix of the variational equation: 

(48) A($«(t,r))= (^|^(x(‘^-"'>)(t),i2(t))+A«(t)|^(x(*^’"'>)(f),w(t)))<i>«(t,r), $«(T,r)=/ 

for all 0 < T < t, A“ being the piecewise continuous solution of the equation 

(49) |Z(^(>J.-o)(i)^^(i)) + A«(t)|^(x(’^-"'>)(t),ii(t)) =0. 

Now introducing ? 7 “(t) = ($“)“^(t, 0)770 = ‘h“(0,t)?7o for an arbitrary 770 7 ^ 0 and setting 

(50) fi^t) 4 
we get the adjoint equation 

(51) r{t) = - (^|/(x(’^^-o)(t),«(t)))'’r7«(t) + ^7rf(t)^^(x(^’"«)(t),72(t)) 
and it can be proven that 

(52) ^{5x{tY'if {t)) = Q yt V 770 7 ^ 0 

According to (14711 . for all perturbation parameters tt, we have thus defined the so-called tangent 
perturbation cone, denoted by /Ct and 77 “ may be interpreted as the normal to the separating 
hyperplane to Xt', moreover /j,“ defined by (ISOll may be interpreted as the Karush-Kuhn-Tucker 
multiplier associated with the constraints g r< 0. The interested reader may refer to [16] or [10] . 

B.2 The iTicLKimum principle 

The following theorem is an adaptation of m Theorem 23, Chapter VI, §35], in the spirit of 
[T2|, using the perturbation cone constructed in the previous section. 

Theorem B.l (Maximum principle) Consider the constrained system {ip, (0), ^ Let 
j.{u,xo) 7 ,g g regular trajectory associated with the piecewise continuous control u € U such that 
a;(“’^”)(ti) £ dRt^{xo) for some ti > 0. Then, there exists a non zero absolutely continuous 77 “ and 
piecewise continuous multipliers 77 “ > 0 , i = 1,... ,p satisfying, for almost all t <ti: 

r{t) = - (^(x(’^’^“)(t),77(t)))^77«(t) +f]/if(t)^^(x(*^’“«)(t),72(t)) 


fix^'"’^°\Tk),Vk) - /(a:("’^“)(rfc),u(rfc)) h 


(53) 
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(54) =0 Vi G p} 

such that, if we define the dualised Hamiltonian 

p 

(55) n{x, u, T], p) = -q^f{x, m) + ^ Pt9i{x, u), 

i=l 

it satisfies 

(56) ma.x'H{x^'^’^°\t),u,q'^{t), p'^{t)) ={t),u{t),q'^{t), p'^{t)) = constant a.e. t<t\ 

uGU 
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